Deep spiking neural networks with applications to human gesture recognition

The spiking neural networks (SNNs), as the 3rd generation of Artificial Neural Networks (ANNs), are a class of event-driven neuromorphic algorithms that potentially have a wide range of application domains and are applicable to a variety of extremely low power neuromorphic hardware. The work presented in this thesis addresses the challenges of human gesture recognition using novel SNN algorithms. It discusses the design of these algorithms for both visual and auditory domain human gesture recognition as well as event-based pre-processing toolkits for audio signals. From the visual gesture recognition aspect, a novel SNN-based event-driven hand gesture recognition system is proposed. This system is shown to be effective in an experiment on hand gesture recognition with its spiking recurrent convolutional neural network (SCRNN) design, which combines both designed convolution operation and recurrent connectivity to maintain spatial and temporal relations with address-event-representation (AER) data. The proposed SCRNN architecture can achieve arbitrary temporal resolution, which means it can exploit temporal correlations between event collections. This design utilises a backpropagation-based training algorithm and does not suffer from gradient vanishing/explosion problems. From the audio perspective, a novel end-to-end spiking speech emotion recognition system (SER) is proposed. This system employs the MFCC as its main speech feature extractor as well as a self-designed latency coding algorithm to effciently convert the raw signal to AER input that can be used for SNN. A two-layer spiking recurrent architecture is proposed to address temporal correlations between spike trains. The robustness of this system is supported by several open public datasets, which demonstrate state of the arts recognition accuracy and a significant reduction in network size, computational costs, and training speed. In addition to directly contributing to neuromorphic SER, this thesis proposes a novel speech-coding algorithm based on the working mechanism of humans auditory organ system. The algorithm mimics the functionality of the cochlea and successfully provides an alternative method of event-data acquisition for audio-based data. The algorithm is then further simplified and extended into an application of speech enhancement which is jointly used in the proposed SER system. This speech-enhancement method uses the lateral inhibition mechanism as a frequency coincidence detector to remove uncorrelated noise in the time-frequency spectrum. The method is shown to be effective by experiments for up to six types of noise.The spiking neural networks (SNNs), as the 3rd generation of Artificial Neural Networks (ANNs), are a class of event-driven neuromorphic algorithms that potentially have a wide range of application domains and are applicable to a variety of extremely low power neuromorphic hardware. The work presented in this thesis addresses the challenges of human gesture recognition using novel SNN algorithms. It discusses the design of these algorithms for both visual and auditory domain human gesture recognition as well as event-based pre-processing toolkits for audio signals. From the visual gesture recognition aspect, a novel SNN-based event-driven hand gesture recognition system is proposed. This system is shown to be effective in an experiment on hand gesture recognition with its spiking recurrent convolutional neural network (SCRNN) design, which combines both designed convolution operation and recurrent connectivity to maintain spatial and temporal relations with address-event-representation (AER) data. The proposed SCRNN architecture can achieve arbitrary temporal resolution, which means it can exploit temporal correlations between event collections. This design utilises a backpropagation-based training algorithm and does not suffer from gradient vanishing/explosion problems. From the audio perspective, a novel end-to-end spiking speech emotion recognition system (SER) is proposed. This system employs the MFCC as its main speech feature extractor as well as a self-designed latency coding algorithm to effciently convert the raw signal to AER input that can be used for SNN. A two-layer spiking recurrent architecture is proposed to address temporal correlations between spike trains. The robustness of this system is supported by several open public datasets, which demonstrate state of the arts recognition accuracy and a significant reduction in network size, computational costs, and training speed. In addition to directly contributing to neuromorphic SER, this thesis proposes a novel speech-coding algorithm based on the working mechanism of humans auditory organ system. The algorithm mimics the functionality of the cochlea and successfully provides an alternative method of event-data acquisition for audio-based data. The algorithm is then further simplified and extended into an application of speech enhancement which is jointly used in the proposed SER system. This speech-enhancement method uses the lateral inhibition mechanism as a frequency coincidence detector to remove uncorrelated noise in the time-frequency spectrum. The method is shown to be effective by experiments for up to six types of noise

A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition

The combination of neuromorphic visual sensors and spiking neural network offers a high efficient bio-inspired solution to real-world applications. However, processing event- based sequences remains challenging because of the nature of their asynchronism and sparsity behavior. In this paper, a novel spiking convolutional recurrent neural network (SCRNN) architecture that takes advantage of both convolution operation and recurrent connectivity to maintain the spatial and temporal relations from event-based sequence data are presented. The use of recurrent architecture enables the network to have a sampling window with an arbitrary length, allowing the network to exploit temporal correlations between event collections. Rather than standard ANN to SNN conversion techniques, the network utilizes a supervised Spike Layer Error Reassignment (SLAYER) training mechanism that allows the network to adapt to neuromorphic (event-based) data directly. The network structure is validated on the DVS gesture dataset and achieves a 10 class gesture recognition accuracy of 96.59% and an 11 class gesture recognition accuracy of 90.28%

Deep convolutional spiking neural network based hand gesture recognition

Novel technologies for EMG (Electromyogram) based hand gesture recognition have been investigated for many industrial applications. In this paper, a novel approach which is based on a specific designed spiking convolution neural network which is fed by a novel EMG signal energy density map is presented. The experimental results indicate that the new approach not only rapidly decreases the required processing time but also increases the average recognition accuracy to 98.76% based on the Strathclyde dataset and to 98.21% based on the CapgMyo open source dataset. A relative comparison of experimental results between the proposed novel EMG based hand gesture recognition methodology and other similar approaches indicates the superior effectiveness of the new design

Noise reduction using neural lateral inhibition for speech enhancement

Recurrent spiking neurons with lateral inhibition connection play a vital role in human’s brain functional abilities. In this paper, we propose a novel noise reduction method that is based on neuron rate coding and bio-inspired spiking neural network architecture. The excitatory-inhibitory topology in the network acts as the temporal characteristic synchrony and coincidence detector that removes uncorrelated noisy spikes. A LIF source encoder is introduced along with the network. The network uses generated binary Short-Time Fourier Transform (STFT) masks according to the rate of processed spike train, which is used to reconstruct the denoised speech signal. The technique is evaluated on noisy speech samples with 5 types of real-world additive noise with different noise strength

Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline

Edge computing solutions that enable the extraction of high level information from a variety of sensors is in increasingly high demand. This is due to the increasing number of smart devices that require sensory processing for their application on the edge. To tackle this problem, we present a smart vision sensor System on Chip (Soc), featuring an event-based camera and a low power asynchronous spiking Convolutional Neuronal Network (sCNN) computing architecture embedded on a single chip. By combining both sensor and processing on a single die, we can lower unit production costs significantly. Moreover, the simple end-to-end nature of the SoC facilitates small stand-alone applications as well as functioning as an edge node in a larger systems. The event-driven nature of the vision sensor delivers high-speed signals in a sparse data stream. This is reflected in the processing pipeline, focuses on optimising highly sparse computation and minimising latency for 9 sCNN layers to $3.36\mu s$. Overall, this results in an extremely low-latency visual processing pipeline deployed on a small form factor with a low energy budget and sensor cost. We present the asynchronous architecture, the individual blocks, the sCNN processing principle and benchmark against other sCNN capable processors

Finishing the euchromatic sequence of the human genome

The sequence of the human genome encodes the genetic instructions for human physiology, as well as rich information about human evolution. In 2001, the International Human Genome Sequencing Consortium reported a draft sequence of the euchromatic portion of the human genome. Since then, the international collaboration has worked to convert this draft into a genome sequence with high accuracy and nearly complete coverage. Here, we report the result of this finishing process. The current genome sequence (Build 35) contains 2.85 billion nucleotides interrupted by only 341 gaps. It covers ∼99% of the euchromatic genome and is accurate to an error rate of ∼1 event per 100,000 bases. Many of the remaining euchromatic gaps are associated with segmental duplications and will require focused work with new methods. The near-complete sequence, the first for a vertebrate, greatly improves the precision of biological analyses of the human genome including studies of gene number, birth and death. Notably, the human enome seems to encode only 20,000-25,000 protein-coding genes. The genome sequence reported here should serve as a firm foundation for biomedical research in the decades ahead

Post-Coronavirus Disease 2019 (COVID-19) Liver Injury in Pregnant Women: A Retrospective Cohort Study

Background: Coronavirus Disease 2019 (COVID-19) has risen as a global threat to public health and can cause both respiratory and multisystemic diseases in humans. This study aimed to describe the incidence of abnormal liver function tests (LFTs) in post-COVID-19 pregnant women, and to explore characteristics of pregnant women with abnormal LFTs. Methods: This retrospective cohort study comprised 155 pregnant patients who experienced COVID-19, alongside 76 uninfected pregnant women as a control group. All participants were randomly selected from the Obstetrics outpatient clinic at the Affiliated Maternity and Child Health Care Hospital of Nantong University between December 25 2022 and January 31 2023. Demographic data and laboratory data were collected, and results were statistically analyzed. Results: Of the 155 pregnant women who had experienced COVID-19, 63 (40.6%) showed abnormally raised liver enzymes. In the control group, 9 (11.8%) cases had abnormal LFTs. Differences between the two groups were statistically significant (p < 0.05). Of the 63 post-COVID-19 patients with abnormal LFTs, the median serum level of alanine transaminase (ALT), aspartate transaminase (AST), and alkaline phosphatase (ALP) was: 175 U/L (range, 51–352 U/L), 113 U/L (range, 42–329 U/L), and 123 U/L (range, 35–250 U/L). Median total biliary acid (TBA) was 18.1 µmol/L (range, 1.8–33.5 µmol/L). The patients who developed abnormal LFTs did so within 7–14 days after contracting COVID-19, with a median of 10 days. Subsequently, their liver function returned to normal within 4–26 days, with a median of 12 days. The univariate analysis on factors that may affect abnormal LFTs revealed a statistically significant difference in gestational age and body mass index (BMI) (p < 0.001). Logistic regression analysis found that gestational age (odds ratio (OR): 1.095 [1.021–1.174]) and BMI (OR: 1.169 [1.059–1.289]) remained a significant independent risk factors for liver injury (p < 0.05). Conclusions: Pregnant women are at an increased risk of liver injury after contracting COVID-19. Moreover, with the increase of gestational age and BMI, the risk of liver injury increases

Non-toxic freezing media to retain the stem cell reserves in adipose tissues

BACKGROUND: Vitamin A and its metabolite, retinoic acid (RA), are important regulators of cell differentiation and organ morphogenesis. Its impact on beef cattle muscle growth remains undefined. METHOD: Angus steer calves were administrated with 0 (control) or 150,000 IU vitamin A (retinyl palmitate in glycerol, i.m.) per calf at birth and 1 month of age. At 2 months of age, a biopsy of the muscle was obtained to analyze the immediate effects of vitamin A injection on myogenic capacity of muscle cells. The resulting steers were harvested at 14 months of age. RESULTS: Vitamin A administration increased cattle growth at 2 months. At 2 months of age, Vitamin A increased PAX7 positive satellite cells and the expression of myogenic marker genes including , , and . Muscle derived mononuclear cells were further isolated and induced myogenesis in vitro. More myotubes and a higher degree of myogenesis was observed in vitamin A groups. Consistently, vitamin A increased (LD) muscle fiber size at harvest. In addition, vitamin A increased the ratio of oxidative type I and type IIA fibers and reduced the glycolic type IIX fibers. Furthermore, we found that RA, a key bioactive metabolite of vitamin A, activated promoter, which explains the upregulated expression of in skeletal muscle. CONCLUSION: Vitamin A administration to neonatal calves enhanced postnatal muscle growth by promoting myogenesis and increasing satellite cell density, accompanied with a shift to oxidative muscle fibers

Effect of dual targeting procyanidins nanoparticles on metabolomics of lipopolysaccharide-stimulated inflammatory macrophages

Inflammation plays an important role in the occurrence and development of many inflammatory diseases. The purpose of this study was to evaluate the anti-inflammatory effect and metabolic behavior of the dual targeting procyanidins (PC) nanoparticles on lipopolysaccharide (LPS)-stimulated inflammatory macrophages by metabolomics method. The double-targeting PC nanoparticles could specifically target both the CD44 receptor and mitochondria, while the single targeting PC-loaded nanoparticles that could target the CD44 receptor on the surface of macrophages. The double-targeting PC nanoparticles had better inhibitory effect than single-targeting PC nanoparticles on the leakage of lactate dehydrogenase and reactive oxygen species overexpression induced by LPS. Amino acid metabolism, energy metabolism and purine metabolism were disordered in LPS-treated group, and metabolic pathway analysis indicated that the double-targeting PC nanoparticles reversed some of LPS impacts. The changes of these potential biomarkers and their corresponding pathways are helpful to further understand the mechanism of PC nanoparticles in alleviating inflammation, and promote their application in nutrition intervention